Cheap Newton Steps for Optimal Control Problems: Automatic Diierentiation and Pantoja's Algorithm

In this paper we discuss Pantoja's construction of the Newton direction for discrete time optimal control problems. We show that Automatic Diierentiation techniques can be used to calculate the Newton direction accurately, without requiring extensive rewriting of user code, and at a surprisingly low computational cost: for an N-step problem with p control variables and q state variables at each step, the worst case cost is 6(p + q + 1) times the computational cost of a single target function evaluation, independent o f N, together with at most p 3 =3 + p 2 (q + 1) + 2 p(q + 1) 2 + (q + 1) 3 , ie less than (p+q+ 1) 3 , oating point m ultiply-and-add operations per timestep. These costs may be considerably reduced if there is signiicant structural sparsity in the problem dynamics. The systematic use of checkpointing roughly doubles the operation counts, but reduces the total space cost to the order of 4pN oating point stores. A n a i v e approach to nding the Newton step would require the solution of an N p N p system of equations together with a number of function evaluations proportional to N p , so this approach to Pantoja's construction is extremely attractive, especially if q is very small relative t o N. Straightforward modiications of the AD algorithms proposed here can be used to implement other discrete time optimal control solution techiniques, such as diierential dynamic programming (DDP), which use state-control feedback. The same techniques also can be used to determine with certainty, at the cost of a single Newton direction calculation, whether or not the Hessian of the target function is suuciently positive deenite at a point o f i n terest. This allows computationally cheap post-hoc veriication that a second-order minimum has been reached to a given accuracy, regardless of what method has been used to obtain it.